Reminding Peer Reviewers of Reporting Guideline Items to Improve Completeness in Published Articles

Key Points Question Does asking peer reviewers to check whether specific reporting guideline items were adequately reported improve the reporting quality in published articles? Findings Two randomized trials were conducted in collaboration with publishing journals. Results from 421 randomly assigned manuscripts (intervention: asking peer reviewers to check specific reporting items; control: usual journal practice) did not indicate any improvement in the reporting quality when peer reviewers were asked to check whether specific reporting guideline items were adequately reported. Meaning These findings indicate that reminding peer reviewers of specific reporting items is not useful in increasing reporting completeness in published articles.


CONSORT-PR:
Inclusion criteria for journals: Journals were eligible for participation if they (i) endorsed the CONSORT statement by mentioning it in the journals' Instruction to authors (ii) had published primary results of at least five RCTs in 2017 (identified using a PubMed search).

Inclusion criteria for manuscripts:
All new manuscript submissions that reported the primary results of RCTs, which the journal editor had decided to send out for external peer review.

Exclusion criterial for manuscripts:
We excluded (i) Manuscripts clearly presenting secondary trial results, additional time points, economic analyses or other analyses (ii) Manuscripts that were clearly labelled as a pilot, feasibility or animal studies (iii) Manuscripts not sent for peer review.

SPIRIT-PR:
Inclusion criteria for journals: It was planned from the beginning that this study will only be conducted at the BMJ Open as it publishes numerous protocols. Hence, no inclusion criteria for the journal were specified.

Inclusion criteria for manuscripts:
All submitted manuscripts sent out for external review that described protocols for RCTs.

Exclusion criterial for manuscripts:
We excluded (i) Manuscripts which were clearly labelled as a pilot or feasibility study (ii) Studies

2)
Sample size (Item 7a) How sample size was determined Is there a clear description of how the sample size was determined, including (1) the estimated outcomes in each group; (2) the α (type I) error level; (3) the statistical power (or the β (type II) error level); and (4) for continuous outcomes, the standard deviation of the measurements?

3)
Sequence generation (Item 8a) Method used to generate random allocation sequence Does the description make it clear if the "assigned intervention is determined by a chance process and cannot be predicted"?

4)
Allocation concealment (Item 9) Mechanism used to implement random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Is it clear how the care provider enrolling participants was made ignorant of the next assignment in the sequence (different from blinding)? Possible methods can rely on centralized or "third-party" assignment (i.e., use of a central telephone randomization system, automated assignment system, sealed containers).

5)
Blinding (Item 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) Is it clear if (1) healthcare providers, (2) patients, and (3) outcome assessors are blinded to the intervention? General terms such as "double-blind" without further specifications should be avoided.

6)
Outcomes and estimation (Item 17a/b) For the primary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence intervals) Is the estimated effect size and its precision (such as standard deviation or 95% confidence intervals) for each treatment arm reported? When the primary outcome is binary, both the relative effect (risk ratio, relative risk) or odds ratio) and the absolute effect (risk difference) should be reported with confidence intervals.

7) Harms (Items 19) All important harms or unintended effects in each group
Is the number of affected persons in each group, the severity grade (if relevant) and the absolute risk (e.g. frequency of incidence) reported? Are the number of serious, life threatening events and deaths reported? If no adverse event occurred this should be clearly stated.

8) Registration (Item 23) Registration number and name of trial registry
Is the registry and the registration number reported? If the trial was not registered, it should be explained why.

9)
Protocol (Item 24) Where trial protocol can be accessed Is it stated where the trial protocol can be assessed (e.g. published, supplementary file, repository, directly from author, confidential and therefore not available)?

10) Funding (Item 25) Sources of funding and other support (such as supply of drugs) and role of funders
Are (1) the funding sources, and (2) the role of the funder(s) described? As described in the study protocol (1), the items were selected using the following approach "For the development of C-short we chose the 10 most important and poorly reported CONSORT items as identified by a group of CONSORT experts in a previous study conducted by Hopewell and colleagues (2). The selection of the items was based on expert opinion and empirical evidence whenever available (2)." The short explanation for each item was "extracted and amended from the CONSORT explanation and elaboration paper (3) and from COBWEB which is an online writing aid tool (2). The short explanation was discussed and adapted by the scientific committee."(1)

SPIRIT-PR
The short version of the SPIRIT checklist was developed using the following steps: (i) we identified SPIRIT items with low adherence in the studies which have assessed adherence to SPIRIT (i.e. Kyte et al (4), ASPIRE-UK (5), and ASPIRE Switzerland, Canada and Germany (ASPIRE-SCAGE (6)); (ii) from these items each member of the steering committee (BS, MB, SH, AO, and AWC) as well, and Dmitry Gryaznov (first author of ASPIRE-SCAGE (6)) selected the 10 most important items; (iii) after one round of consensus the 10 most important and poorly reported SPIRIT items were identified. The sub-items of these 10 items which we focused on in the intervention were discussed amongst the steering committee.
The key features as bullet points for each item were developed in close adherence to the SPIRIT explanation and elaboration paper (7). We hypothesized that the mean adherence in the control group would be 71% (based on recent evidence (2) and considering that the items "Registration", "Protocol", and "Funding" are thoroughly checked in PLOS journals given their policy requirement; see protocol for more details (1)). Using a twosided t-test, a sample size of 166 (83 per treatment arm) had a power of 80% to find a 15% relative increase (71% adherence in control group; 82% adherence in intervention group; standard deviation [SD] =0.25; a type 1 error at 5%). Recruitment was stopped when it was clear that sufficient manuscripts were enrolled to reach the 83 published articles (i.e. not randomized articles) per treatment arm.

SPIRIT-PR
Between November and December 2019 we selected randomly 20 eligible published RCT protocols that were published in BMJ Open. The mean adherence to the SPIRIT reporting guideline for the 10 selected most important and poorly reported SPIRIT items was assessed to be 53% (SD 0.20). Using a two-sided t-test (type I error 5%; power 90%) a sample size of 88 was required to show a relative increase of 25% (intervention: mean adherence of 67% vs 53% in control group). To be conservative and allow for a higher variance of the data (SD 0.22) we increased the sample size to 106 (53 per treatment arm). Recruitment was stopped when it was clear that sufficient manuscripts were enrolled to reach the 53 published articles (i.e. not randomized articles) per treatment arm.  Sample size (14) Estimated number of participants needed to achieve study objectives and how it was determined, including clinical and statistical assumptions supporting any sample size calculations .
• Total and per group sample size should be clear (including α-value and power) • Outcome used for sample size calculation should be mentioned • The values assumed for the outcome in each study group or for one study group plus effect size • A rationale or reference for the outcome values assumed for each study group • Type of statistical test provided • Adjustment for losses to follow-up, if relevant, described 3 Recruitment (15) Strategies for achieving adequate participant enrolment to reach target sample size.
• Location of recruitment described (e.g. emergency department, community) • Person identifying patients described (e.g. GP, surgeon, study nurse) • Expected recruitment rate or duration or recruitment stated 4 Allocation implementation (16 c) Who will generate the allocation sequence, who will enrol participants, and who will assign participants to interventions.
• Person who will enrol/assign participants described 5 Blinding (17 a) Who will be blinded after assignment to interventions (eg, trial participants, care providers, outcome assessors, data analysts) and how .
• Blinding status of trial participants clear • Blinding status of care providers clear • Blinding status of outcome assessor for primary outcome clear 6 Data collection methods (18 a) Plans for assessment and collection of outcome, baseline, and other trial data, including any related processes to promote data quality (eg, duplicate measurements, training of assessors) and a description of study instruments (eg, questionnaires, laboratory tests) along with their reliability and validity, if known. Reference to where data collection forms can be found, if not in the protocol.
• Personnel who will collect data for the primary outcome 7 Data collection methodsretention (18 b) Plans to promote participant retention and complete follow-up, including list of any outcome data to be collected for participants who discontinue or deviate from intervention protocols.
• Strategies to promote participant retention and complete follow up (e.g. phone call reminders, financial compensation) 8 Statistical methods (20 a) Statistical methods for analysing the primary outcome. Reference to where other details of the statistical analysis plan can be found, if not in the protocol .
• The main analysis of the primary outcome, including the analysis methods to be used for statistical comparisons • Effect measure for primary outcome specified (e.g. OR, RR, difference in means) • Significance level and/or intended use of confidence intervals (at least one of this two) specified 9 Population analysed (20 c) Definition of analysis population relating to protocol non-adherence (eg, as randomised analysis), and any statistical methods to handle missing data (eg, multiple imputation).